1.2.1监督学习

1.2.1 Supervised Learning

在监督学习1中，数据集是标记示例的集合 $\{(\mathbf{x}_i, y_i)\}_{i=1}^N$。每个元素$\mathbf{x}_i$之中$氮$称为特征向量。特征向量是一个向量，其中每个维度$j=1,\ldots,D$包含一个以某种方式描述示例的值。该值称为特征，表示为$x^{(j)}$。例如，如果每个示例$\mathbf{x}$在我们的集合中代表一个人，那么第一个特征，$x^{(1)}$，可以包含以厘米为单位的高度，第二个特征，$x^{(2)}$，可以包含以公斤为单位的重量，$x^{(3)}$可以包含性别等。对于数据集中的所有示例，位置处的特征$j$特征向量中总是包含相同类型的信息。这意味着如果$x_i^{(2)}$在某些示例中包含以千克为单位的重量$\mathbf{x}_i$， 然后$x_k^{(2)}$每个示例中还将包含以千克为单位的重量$\mathbf{x}_k$,$k=1,\l点,N$。标签​ $义$可以是属于有限类集的元素 $\{1,2,\l点,C\}$，或实数，或更复杂的结构，如向量、矩阵、树或图。除非另有说明，本书中$义$是有限类集合之一或实数2。您可以将类视为示例所属的类别。例如，如果您的示例是电子邮件，而您的问题是垃圾邮件检测，那么您有两个类$\{垃圾邮件，不是\_垃圾邮件\}$。

In supervised learning1, the dataset is the collection of labeled examples $\{(\mathbf{x}_i, y_i)\}_{i=1}^N$. Each element $\mathbf{x}_i$ among $N$ is called a feature vector. A feature vector is a vector in which each dimension $j=1,\ldots,D$ contains a value that describes the example somehow. That value is called a feature and is denoted as $x^{(j)}$. For instance, if each example $\mathbf{x}$ in our collection represents a person, then the first feature, $x^{(1)}$, could contain height in cm, the second feature, $x^{(2)}$, could contain weight in kg, $x^{(3)}$ could contain gender, and so on. For all examples in the dataset, the feature at position $j$ in the feature vector always contains the same kind of information. It means that if $x_i^{(2)}$ contains weight in kg in some example $\mathbf{x}_i$, then $x_k^{(2)}$ will also contain weight in kg in every example $\mathbf{x}_k$, $k=1,\ldots,N$. The label $y_i$ can be either an element belonging to a finite set of classes $\{1,2,\ldots,C\}$, or a real number, or a more complex structure, like a vector, a matrix, a tree, or a graph. Unless otherwise stated, in this book $y_i$ is either one of a finite set of classes or a real number2. You can see a class as a category to which an example belongs. For instance, if your examples are email messages and your problem is spam detection, then you have two classes $\{spam, not\_spam\}$.

监督学习算法的目标是使用数据集生成采用特征向量的模型$\mathbf{x}$作为输入和输出信息，可以推断出该特征向量的标签。例如，使用人员数据集创建的模型可以将描述人员的特征向量作为输入，并输出该人员患有癌症的概率。

The goal of a supervised learning algorithm is to use the dataset to produce a model that takes a feature vector $\mathbf{x}$ as input and outputs information that allows deducing the label for this feature vector. For instance, the model created using the dataset of people could take as input a feature vector describing a person and output a probability that the person has cancer.

1.2.2无监督学习

1.2.2 Unsupervised Learning

在无监督学习中，数据集是未标记示例的集合。 $\{\mathbf{x}_i\}_{i=1}^N$。再次，$\mathbf{x}$是一个特征向量，无监督学习算法的目标是创建一个采用特征向量的模型$\mathbf{x}$作为输入，并将其转换为另一个向量或可用于解决实际问题的值。例如，在聚类中，模型返回数据集中每个特征向量的聚类 ID。在降维中，模型的输出是特征向量，其特征少于输入。$\mathbf{x}$;在异常值检测中，输出是一个实数，表明如何$\mathbf{x}$与数据集中的“典型”示例不同。

In unsupervised learning, the dataset is a collection of unlabeled examples $\{\mathbf{x}_i\}_{i=1}^N$. Again, $\mathbf{x}$ is a feature vector, and the goal of an unsupervised learning algorithm is to create a model that takes a feature vector $\mathbf{x}$ as input and either transforms it into another vector or into a value that can be used to solve a practical problem. For example, in clustering, the model returns the id of the cluster for each feature vector in the dataset. In dimensionality reduction, the output of the model is a feature vector that has fewer features than the input $\mathbf{x}$; in outlier detection, the output is a real number that indicates how $\mathbf{x}$ is different from a “typical” example in the dataset.

1.2.3半监督学习

1.2.3 Semi-Supervised Learning

在半监督学习中，数据集包含标记和未标记的示例。通常，未标记示例的数量远高于标记示例的数量。半监督学习算法的目标与监督学习算法的目标相同。这里的希望是，使用许多未标记的示例可以帮助学习算法找到（我们可能会说“产生”或“计算”）更好的模型。

In semi-supervised learning, the dataset contains both labeled and unlabeled examples. Usually, the quantity of unlabeled examples is much higher than the number of labeled examples. The goal of a semi-supervised learning algorithm is the same as the goal of the supervised learning algorithm. The hope here is that using many unlabeled examples can help the learning algorithm to find (we might say “produce” or “compute”) a better model.

添加更多未标记的示例可以使学习受益，这看起来可能违反直觉。看来我们给问题增加了更多的不确定性。但是，当您添加未标记的示例时，您会添加有关问题的更多信息：更大的样本可以更好地反映我们标记的数据来自的概率分布。理论上，学习算法应该能够利用这些附加信息。

It could look counter-intuitive that learning could benefit from adding more unlabeled examples. It seems like we add more uncertainty to the problem. However, when you add unlabeled examples, you add more information about your problem: a larger sample reflects better the probability distribution the data we labeled came from. Theoretically, a learning algorithm should be able to leverage this additional information.

1.2.4强化学习

1.2.4 Reinforcement Learning

强化学习是机器学习的一个子领域，机器“生活”在一个环境中，并且能够将该环境的状态感知为特征向量。机器可以在每种状态下执行操作。不同的行为会带来不同的奖励，也可能使机器进入另一种环境状态。强化学习算法的目标是学习策略。

Reinforcement learning is a subfield of machine learning where the machine “lives” in an environment and is capable of perceiving the state of that environment as a vector of features. The machine can execute actions in every state. Different actions bring different rewards and could also move the machine to another state of the environment. The goal of a reinforcement learning algorithm is to learn a policy.

策略是一个函数（类似于监督学习中的模型），它将状态的特征向量作为输入并输出在该状态下执行的最佳操作。如果该行动使预期平均奖励最大化，则该行动是最优的。

A policy is a function (similar to the model in supervised learning) that takes the feature vector of a state as input and outputs an optimal action to execute in that state. The action is optimal if it maximizes the expected average reward.

强化学习解决特定类型的问题，其中决策是连续的，并且目标是长期的，例如游戏、机器人、资源管理或物流。在本书中，我强调一次性决策，其中输入示例与过去做出的预测相互独立。我将强化学习排除在本书的讨论范围之外。

Reinforcement learning solves a particular kind of problem where decision making is sequential, and the goal is long-term, such as game playing, robotics, resource management, or logistics. In this book, I put emphasis on one-shot decision making where input examples are independent of one another and the predictions made in the past. I leave reinforcement learning out of the scope of this book.

2.1.1数据结构

2.1.1 Data Structures

标量是一个简单的数值，例如$15$或者$-3.25$。采用标量值的变量或常量用斜体字母表示，例如$X$或者$A$。

A scalar is a simple numerical value, like $15$ or $-3.25$. Variables or constants that take scalar values are denoted by an italic letter, like $x$ or $a$.

向量是标量值的有序列表，称为属性。我们将向量表示为粗体字符，例如，$\mathbf{x}$或者$\mathbf{w}$。向量可以可视化为指向某些方向的箭头以及多维空间中的点。三个二维向量的插图，$\mathbf{a}=[2,3]$,$\mathbf{b}=[-2,5]$， 和$\mathbf{c}=[1,0]$在图中给出。  2和图。  3 .我们将向量的属性表示为带有索引的斜体值，如下所示：$w^{(j)}$或者$x^{(j)}$。指数$j$表示向量的特定维度，即属性在列表中的位置。例如，在向量中$\mathbf{a}$如图中红色所示 2和图。  3 ,$a^{(1)} = 2$和$a^{(2)} = 3$。

A vector is an ordered list of scalar values, called attributes. We denote a vector as a bold character, for example, $\mathbf{x}$ or $\mathbf{w}$. Vectors can be visualized as arrows that point to some directions as well as points in a multi-dimensional space. Illustrations of three two-dimensional vectors, $\mathbf{a}=[2,3]$, $\mathbf{b}=[-2,5]$, and $\mathbf{c}=[1,0]$ are given in fig. 2 and fig. 3. We denote an attribute of a vector as an italic value with an index, like this: $w^{(j)}$ or $x^{(j)}$. The index $j$ denotes a specific dimension of the vector, the position of an attribute in the list. For instance, in the vector $\mathbf{a}$ shown in red in fig. 2 and fig. 3, $a^{(1)} = 2$ and $a^{(2)} = 3$.

符号$x^{(j)}$不应与幂运算符混淆，例如$2$在$x^2$（平方）或$3$在$x^3$（立方）。如果我们想将幂运算符（例如平方）应用于向量的索引属性，我们可以这样写：$(x^{(j)})^2$。

The notation $x^{(j)}$ should not be confused with the power operator, such as the $2$ in $x^2$ (squared) or $3$ in $x^3$ (cubed). If we want to apply a power operator, say square, to an indexed attribute of a vector, we write like this: $(x^{(j)})^2$.

一个变量可以有两个或多个索引，如下所示：$x_i^{(j)}$或者像这样$x_{i,j}^{(k)}$。例如，在神经网络中，我们表示为$x_{l,u}^{(j)}$输入特征$j$单位的$你$层内$我$。

A variable can have two or more indices, like this: $x_i^{(j)}$ or like this $x_{i,j}^{(k)}$. For example, in neural networks, we denote as $x_{l,u}^{(j)}$ the input feature $j$ of unit $u$ in layer $l$.

矩阵是按行和列排列的数字的矩形阵列。下面是一个两行三列的矩阵示例，

A matrix is a rectangular array of numbers arranged in rows and columns. Below is an example of a matrix with two rows and three columns,

$$\begin{bmatrix}2&4&-3\\21&-6&-1\end{bmatrix}。$$

$$\begin{bmatrix}2&4&-3\\21&-6&-1\end{bmatrix}.$$

矩阵用粗体大写字母表示，例如$\mathbf{A}$或者$\mathbf{W}$。

Matrices are denoted with bold capital letters, such as $\mathbf{A}$ or $\mathbf{W}$.

集合是唯一元素的无序集合。我们将集合表示为书法大写字符，例如，$\数学{S}$。一组数字可以是有限的（包括固定数量的值）。在这种情况下，它使用荣誉来表示，例如，$\{1,3,18,23,235\}$或者$\{x_1,x_2,x_3,x_4,\l点,x_n\}$。集合可以是无限的，并且包括某个区间内的所有值。如果一个集合包含之间的所有值$A$和$乙$， 包括$A$和$乙$，用括号表示为$[一，二]$。如果集合不包含值$A$和$乙$，这样的集合使用括号表示，如下所示：$(一、二)$。例如，集合$[0,1]$包括这样的值$0$,$0.0001$,$0.25$,$0.784$,$0.9995$， 和$1.0$。一个特殊的集合表示$\mathbb{R}$包括从负无穷大到正无穷大的所有数字。

A set is an unordered collection of unique elements. We denote a set as a calligraphic capital character, for example, $\mathcal{S}$. A set of numbers can be finite (include a fixed amount of values). In this case, it is denoted using accolades, for example, $\{1,3,18,23,235\}$ or $\{x_1,x_2,x_3,x_4,\ldots,x_n\}$. A set can be infinite and include all values in some interval. If a set includes all values between $a$ and $b$, including $a$ and $b$, it is denoted using brackets as $[a,b]$. If the set doesn’t include the values $a$ and $b$, such a set is denoted using parentheses like this: $(a,b)$. For example, the set $[0,1]$ includes such values as $0$, $0.0001$, $0.25$, $0.784$, $0.9995$, and $1.0$. A special set denoted $\mathbb{R}$ includes all numbers from minus infinity to plus infinity.

当一个元素$X$属于一个集合$\数学{S}$， 我们写$x \in \mathcal{S}$。我们可以获得一套新的$\mathcal{S}_3$作为两个集合的交集$\mathcal{S}_1$和$\mathcal{S}_2$。在这种情况下，我们写$\mathcal{S}_3 \leftarrow \mathcal{S}_1 \cap \mathcal{S}_2$。例如$\{1,3,5,8\} \cap \{1,8,4\}$给出新的集合$\{1,8\}$。

When an element $x$ belongs to a set $\mathcal{S}$, we write $x \in \mathcal{S}$. We can obtain a new set $\mathcal{S}_3$ as an intersection of two sets $\mathcal{S}_1$ and $\mathcal{S}_2$. In this case, we write $\mathcal{S}_3 \leftarrow \mathcal{S}_1 \cap \mathcal{S}_2$. For example $\{1,3,5,8\} \cap \{1,8,4\}$ gives the new set $\{1,8\}$.

我们可以获得一套新的$\mathcal{S}_3$作为两个集合的并集$\mathcal{S}_1$和$\mathcal{S}_2$。在这种情况下，我们写$\mathcal{S}_3 \leftarrow \mathcal{S}_1 \cup \mathcal{S}_2$。例如$\{1,3,5,8\} \cup \{1,8,4\}$给出新的集合$\{1,3,4,5,8\}$。

We can obtain a new set $\mathcal{S}_3$ as a union of two sets $\mathcal{S}_1$ and $\mathcal{S}_2$. In this case, we write $\mathcal{S}_3 \leftarrow \mathcal{S}_1 \cup \mathcal{S}_2$. For example $\{1,3,5,8\} \cup \{1,8,4\}$ gives the new set $\{1,3,4,5,8\}$.

2.1.2大写西格玛符号

2.1.2 Capital Sigma Notation

对集合的求和$\mathit{X} = \{x_1, x_2, \ldots, x_{n-1}, x_n\}$或向量的属性$\mathbf{x} = [x^{(1)}, x^{(2)}, \ldots, x^{(m-1)}, x^{(m)}]$表示如下：

The summation over a collection $\mathit{X} = \{x_1, x_2, \ldots, x_{n-1}, x_n\}$ or over the attributes of a vector $\mathbf{x} = [x^{(1)}, x^{(2)}, \ldots, x^{(m-1)}, x^{(m)}]$ is denoted like this:

$$\sum_{i=1}^n x_i \stackrel{\text{def}}{=} x_1 + x_2 + \ldots + x_{n-1} + x_n,$$

$$\sum_{i=1}^n x_i \stackrel{\text{def}}{=} x_1 + x_2 + \ldots + x_{n-1} + x_n,$$

要不然：

or else:

$$\sum_{j=1}^mx^{(j)} \stackrel{\text{def}}{=} x^{(1)} + x^{(2)} + \ldots + x^{( m-1)} + x^{(m)}。$$

$$\sum_{j=1}^m x^{(j)} \stackrel{\text{def}}{=} x^{(1)} + x^{(2)} + \ldots + x^{(m-1)} + x^{(m)}.$$

符号$\stackrel{\text{def}}{=}$意思是“被定义为”。

The notation $\stackrel{\text{def}}{=}$ means “is defined as”.

2.1.3大写 Pi 表示法

2.1.3 Capital Pi Notation

类似于大写 sigma 的符号是大写 pi 符号。它表示集合中元素或向量属性的乘积：

A notation analogous to capital sigma is the capital pi notation. It denotes a product of elements in a collection or attributes of a vector:

$$\prod_{i=1}^n x_i \stackrel{\text{def}}{=} x_1 \cdot x_2 \cdot \ldots \cdot x_{n-1} \cdot x_n,$$

$$\prod_{i=1}^n x_i \stackrel{\text{def}}{=} x_1 \cdot x_2 \cdot \ldots \cdot x_{n-1} \cdot x_n,$$

在哪里$a \cdot b$方法$a$乘以$b$。在可能的情况下，我们省略$\cdot$为了简化符号，所以$a b$也意味着$a$乘以$b$。

where $a \cdot b$ means $a$ multiplied by $b$. Where possible, we omit $\cdot$ to simplify the notation, so $a b$ also means $a$ multiplied by $b$.

2.1.4集合运算

2.1.4 Operations on Sets

派生集创建运算符如下所示：$\mathcal{S}' \leftarrow \{x^2\, |\, x \in \mathcal{S}, x > 3 \}$。这个符号意味着我们创建一个新集合$\mathcal{S}'$通过放入其中$x$平方使得$x$是在$\mathcal{S}$， 和$x$大于$3$。

A derived set creation operator looks like this: $\mathcal{S}' \leftarrow \{x^2\, |\, x \in \mathcal{S}, x > 3 \}$. This notation means that we create a new set $\mathcal{S}'$ by putting into it $x$ squared such that that $x$ is in $\mathcal{S}$, and $x$ is greater than $3$.

基数运算符$|\mathcal{S}|$返回集合中元素的数量$\mathcal{S}$。

The cardinality operator $|\mathcal{S}|$ returns the number of elements in set $\mathcal{S}$.

2.1.5向量运算

2.1.5 Operations on Vectors

两个向量的和$\mathbf{x} + \mathbf{z}$定义为向量$[x^{(1)} + z^{(1)}, x^{(2)} + z^{(2)}, \ldots, x^{(m)} + z^{(m)}]$。

The sum of two vectors $\mathbf{x} + \mathbf{z}$ is defined as the vector $[x^{(1)} + z^{(1)}, x^{(2)} + z^{(2)}, \ldots, x^{(m)} + z^{(m)}]$.

两个向量的差$\mathbf{x} - \mathbf{z}$定义为$[x^{(1)} - z^{(1)}, x^{(2)} - z^{(2)}, \ldots, x^{(m)} - z^{(m)}]$。

The difference of two vectors $\mathbf{x} - \mathbf{z}$ is defined as $[x^{(1)} - z^{(1)}, x^{(2)} - z^{(2)}, \ldots, x^{(m)} - z^{(m)}]$.

向量乘以标量就是向量。例如$\mathbf{x}c \stackrel{\text{def}}{=} [cx^{(1)}, cx^{(2)},\ldots,cx^{(m)}]$。

A vector multiplied by a scalar is a vector. For example $\mathbf{x}c \stackrel{\text{def}}{=} [cx^{(1)}, cx^{(2)},\ldots,cx^{(m)}]$.

两个向量的点积是标量。例如，$\mathbf{w}\mathbf{x} \stackrel{\text{def}}{=} \sum_{i=1}^m w^{(i)}x^{(i)}$。在一些书中，点积表示为$\mathbf{w}\cdot\mathbf{x}$。两个向量必须具有相同的维度。否则，点积未定义。

A dot-product of two vectors is a scalar. For example, $\mathbf{w}\mathbf{x} \stackrel{\text{def}}{=} \sum_{i=1}^m w^{(i)}x^{(i)}$. In some books, the dot-product is denoted as $\mathbf{w}\cdot\mathbf{x}$. The two vectors must be of the same dimensionality. Otherwise, the dot-product is undefined.

矩阵的乘法$\mathbf{W}$通过向量$\mathbf{x}$结果产生另一个向量。让我们的矩阵是，

The multiplication of a matrix $\mathbf{W}$ by a vector $\mathbf{x}$ results in another vector. Let our matrix be,

$$\mathbf{W} = \begin{bmatrix} w^{(1,1)} & w^{(1,2)} & w^{(1,3)} \\ w^{(2,1)} & w^{(2,2)} & w^{(2,3)} \end{bmatrix}.$$

$$\mathbf{W} = \begin{bmatrix} w^{(1,1)} & w^{(1,2)} & w^{(1,3)} \\ w^{(2,1)} & w^{(2,2)} & w^{(2,3)} \end{bmatrix}.$$

当向量参与矩阵运算时，向量默认表示为一列矩阵。当向量位于矩阵右侧时，它仍然是列向量。仅当向量的行数与矩阵的列数相同时，我们才能将矩阵乘以向量。设我们的向量为$\mathbf{x} \stackrel{\text{def}}{=} [x^{(1)},x^{(2)},x^{(3)}]$。然后$\mathbf{W}\mathbf{x}$是一个二维向量，定义为，

When vectors participate in operations on matrices, a vector is by default represented as a matrix with one column. When the vector is on the right of the matrix, it remains a column vector. We can only multiply a matrix by vector if the vector has the same number of rows as the number of columns in the matrix. Let our vector be $\mathbf{x} \stackrel{\text{def}}{=} [x^{(1)},x^{(2)},x^{(3)}]$. Then $\mathbf{W}\mathbf{x}$ is a two-dimensional vector defined as,

$$\begin{align*} \mathbf{W}\mathbf{x} &= \begin{bmatrix} w^{(1,1)} & w^{(1,2)} & w^{(1,3)} \\ w^{(2,1)} & w^{(2,2)} & w^{(2,3)} \end{bmatrix} \begin{bmatrix} x^{(1)}\\ x^{(2)}\\ x^{(3)} \end{bmatrix}\\ &\stackrel{\text{def}}{=} \begin{bmatrix} w^{(1,1)}x^{(1)} + w^{(1,2)}x^{(2)} + w^{(1,3)}x^{(3)}\\ w^{(2,1)}x^{(1)} + w^{(2,2)}x^{(2)} + w^{(2,3)}x^{(3)} \end{bmatrix}\\ &= \begin{bmatrix} \mathbf{w}^{(1)}\mathbf{x}\\ \mathbf{w}^{(2)}\mathbf{x} \end{bmatrix} \end{align*}$$

$$\begin{align*} \mathbf{W}\mathbf{x} &= \begin{bmatrix} w^{(1,1)} & w^{(1,2)} & w^{(1,3)} \\ w^{(2,1)} & w^{(2,2)} & w^{(2,3)} \end{bmatrix} \begin{bmatrix} x^{(1)}\\ x^{(2)}\\ x^{(3)} \end{bmatrix}\\ &\stackrel{\text{def}}{=} \begin{bmatrix} w^{(1,1)}x^{(1)} + w^{(1,2)}x^{(2)} + w^{(1,3)}x^{(3)}\\ w^{(2,1)}x^{(1)} + w^{(2,2)}x^{(2)} + w^{(2,3)}x^{(3)} \end{bmatrix}\\ &= \begin{bmatrix} \mathbf{w}^{(1)}\mathbf{x}\\ \mathbf{w}^{(2)}\mathbf{x} \end{bmatrix} \end{align*}$$

如果我们的矩阵有五行，那么乘积的结果将是一个五维向量。

If our matrix had, say, five rows, the result of the product would be a five-dimensional vector.

当向量在乘法中位于矩阵的左侧时，在与矩阵相乘之前必须对其进行转置。向量的转置$\mathbf{x}$表示为$\mathbf{x}^\top$从列向量中生成行向量。比方说，

When the vector is on the left side of the matrix in the multiplication, then it has to be transposed before we multiply it by the matrix. The transpose of the vector $\mathbf{x}$ denoted as $\mathbf{x}^\top$ makes a row vector out of a column vector. Let’s say,

$$\mathbf{x} = \begin{bmatrix} x^{(1)}\\ x^{(2)} \end{bmatrix},\, \, \textrm{then}\, \, \mathbf{x}^\top \stackrel{\text{def}}{=} \begin{bmatrix} x^{(1)} & x^{(2)} \end{bmatrix}.$$

$$\mathbf{x} = \begin{bmatrix} x^{(1)}\\ x^{(2)} \end{bmatrix},\, \, \textrm{then}\, \, \mathbf{x}^\top \stackrel{\text{def}}{=} \begin{bmatrix} x^{(1)} & x^{(2)} \end{bmatrix}.$$

向量的乘法$\mathbf{x}$由矩阵$\mathbf{W}$是（谁）给的$\mathbf{x}^{\top}\mathbf{W}$,

The multiplication of the vector $\mathbf{x}$ by the matrix $\mathbf{W}$ is given by $\mathbf{x}^{\top}\mathbf{W}$,

$$\begin{align*} \mathbf{x}^{\top}\mathbf{W} &= \begin{bmatrix} x^{(1)} & x^{(2)} \end{bmatrix} \begin{bmatrix} w^{(1,1)} & w^{(1,2)} & w^{(1,3)} \\ w^{(2,1)} & w^{(2,2)} & w^{(2,3)} \end{bmatrix}\\ &\stackrel{\text{def}}{=} \big[ w^{(1,1)}x^{(1)} + w^{(2,1)}x^{(2)},\\ &\,\,\,\,\,\, w^{(1,2)}x^{(1)} + w^{(2,2)}x^{(2)}, \\ &\,\,\,\,\,\, w^{(1,3)}x^{(1)} + w^{(2,3)}x^{(2)}\big] \end{align*}$$

$$\begin{align*} \mathbf{x}^{\top}\mathbf{W} &= \begin{bmatrix} x^{(1)} & x^{(2)} \end{bmatrix} \begin{bmatrix} w^{(1,1)} & w^{(1,2)} & w^{(1,3)} \\ w^{(2,1)} & w^{(2,2)} & w^{(2,3)} \end{bmatrix}\\ &\stackrel{\text{def}}{=} \big[ w^{(1,1)}x^{(1)} + w^{(2,1)}x^{(2)},\\ &\,\,\,\,\,\, w^{(1,2)}x^{(1)} + w^{(2,2)}x^{(2)}, \\ &\,\,\,\,\,\, w^{(1,3)}x^{(1)} + w^{(2,3)}x^{(2)}\big] \end{align*}$$

正如您所看到的，只有当向量的维数与矩阵的行数相同时，我们才能将向量乘以矩阵。

As you can see, we can only multiply a vector by a matrix if the vector has the same number of dimensions as the number of rows in the matrix.

2.1.6功能

2.1.6 Functions

函数是关联每个元素的关系$x$一组的$\mathcal{X}$，函数的域，到单个元素$y$另一组的$\mathcal{Y}$，函数的余域。函数通常有一个名称。如果该函数被调用$f$，这个关系表示为$y = f(x)$（读$f$的$x$), 元素$x$是函数的参数或输入，并且$y$是函数或输出的值。用于表示输入的符号是函数的变量（我们常说$f$是变量的函数$x$）。

A function is a relation that associates each element $x$ of a set $\mathcal{X}$, the domain of the function, to a single element $y$ of another set $\mathcal{Y}$, the codomain of the function. A function usually has a name. If the function is called $f$, this relation is denoted $y = f(x)$ (read $f$ of $x$), the element $x$ is the argument or input of the function, and $y$ is the value of the function or the output. The symbol that is used for representing the input is the variable of the function (we often say that $f$ is a function of the variable $x$).

我们这么说$f(x)$局部最小值为$x = c$如果$f(x) \geq f(c)$对于每一个$x$在一些开区间内$x = c$。区间是一组实数，其属性是位于该集合中两个数字之间的任何数字也包含在该集合中。开区间不包括其端点，并使用括号表示。例如，$(0,1)$意思是“所有大于$0$并且小于$1$”。所有局部最小值中的最小值称为全局最小值。参见图 1 中的插图。  4 .

We say that $f(x)$ has a local minimum at $x = c$ if $f(x) \geq f(c)$ for every $x$ in some open interval around $x = c$. An interval is a set of real numbers with the property that any number that lies between two numbers in the set is also included in the set. An open interval does not include its endpoints and is denoted using parentheses. For example, $(0,1)$ means “all numbers greater than $0$ and less than $1$”. The minimal value among all the local minima is called the global minimum. See illustration in fig. 4.

向量函数，表示为$\mathbf{y} = \bm{f}(x)$是一个返回向量的函数$\mathbf{y}$。它可以有一个向量或一个标量参数。

A vector function, denoted as $\mathbf{y} = \bm{f}(x)$ is a function that returns a vector $\mathbf{y}$. It can have a vector or a scalar argument.

2.1.7最大值和精氨酸最大值

2.1.7 Max and Arg Max

给定一组值$\mathcal{A} = \{a_1, a_2,\ldots, a_n\}$， 运营商$\max_{a\in\mathcal{A}} f(a)$返回最高值$f(a)$对于集合中的所有元素$\mathcal{A}$。另一方面，运营商$\underset{a\in\mathcal{A}}{\mathrm{argmax}}f(a)$返回集合的元素$\mathcal{A}$最大化$f(a)$。

Given a set of values $\mathcal{A} = \{a_1, a_2,\ldots, a_n\}$, the operator $\max_{a\in\mathcal{A}} f(a)$ returns the highest value $f(a)$ for all elements in the set $\mathcal{A}$. On the other hand, the operator $\underset{a\in\mathcal{A}}{\mathrm{argmax}}f(a)$ returns the element of the set $\mathcal{A}$ that maximizes $f(a)$.

有时，当集合是隐式的或无限的时，我们可以写$\max_{a} f(a)$或者$\underset{a}{\mathrm{argmax}}f(a)$。

Sometimes, when the set is implicit or infinite, we can write $\max_{a} f(a)$ or $\underset{a}{\mathrm{argmax}}f(a)$.

运营商$\min$和$\mathrm{argmin}$以类似的方式操作。

Operators $\min$ and $\mathrm{argmin}$ operate in a similar manner.

2.1.8赋值运算符

2.1.8 Assignment Operator

表达方式$a \gets f(x)$意味着变量$a$得到新值：结果$f(x)$。我们说变量$a$被分配一个新值。相似地，$\mathbf{a} \gets [a_1, a_2]$意味着向量变量$\mathbf{a}$获取二维向量值$[a_1, a_2]$。

The expression $a \gets f(x)$ means that the variable $a$ gets the new value: the result of $f(x)$. We say that the variable $a$ gets assigned a new value. Similarly, $\mathbf{a} \gets [a_1, a_2]$ means that the vector variable $\mathbf{a}$ gets the two-dimensional vector value $[a_1, a_2]$.

2.1.9导数和梯度

2.1.9 Derivative and Gradient

衍生品 $f'$一个函数的$f$是一个函数或一个值，描述多快$f$增长（或减少）。如果导数是一个常数值，例如$5$或者$-3$，则函数在任意点不断增大（或减小）$x$其域。如果导数$f'$是一个函数，那么函数$f$可以在其领域的不同区域以不同的速度增长。如果导数$f'$在某个时刻是积极的$x$，那么函数$f$在这一点上成长。如果导数为$f$在某些情况下是负数$x$，则此时函数减小。零处的导数$x$意味着函数的斜率在$x$是水平的。

A derivative $f'$ of a function $f$ is a function or a value that describes how fast $f$ grows (or decreases). If the derivative is a constant value, like $5$ or $-3$, then the function grows (or decreases) constantly at any point $x$ of its domain. If the derivative $f'$ is a function, then the function $f$ can grow at a different pace in different regions of its domain. If the derivative $f'$ is positive at some point $x$, then the function $f$ grows at this point. If the derivative of $f$ is negative at some $x$, then the function decreases at this point. The derivative of zero at $x$ means that the function’s slope at $x$ is horizontal.

寻找导数的过程称为微分。

The process of finding a derivative is called differentiation.

基本函数的导数是已知的。例如如果$f(x) = x^2$， 然后$f'(x) = 2x$;如果$f(x) = 2x$然后$f'(x) = 2$;如果$f(x) = 2$然后$f'(x) = 0$（任何函数的导数$f(x) = c$， 在哪里$c$是一个常数，为零）。

Derivatives for basic functions are known. For example if $f(x) = x^2$, then $f'(x) = 2x$; if $f(x) = 2x$ then $f'(x) = 2$; if $f(x) = 2$ then $f'(x) = 0$ (the derivative of any function $f(x) = c$, where $c$ is a constant value, is zero).

如果我们要求导的函数不是基函数，我们可以利用链式法则求它的导数。例如如果$F(x) = f(g(x))$， 在哪里$f$和$g$是一些函数，那么$F'(x)=f'(g(x))g'(x)$。例如如果$F(x) = (5x + 1)^2$然后$g(x) = 5x + 1$和$f(g(x)) = (g(x))^2$。应用链式法则，我们发现：

If the function we want to differentiate is not basic, we can find its derivative using the chain rule. For instance if $F(x) = f(g(x))$, where $f$ and $g$ are some functions, then $F'(x)=f'(g(x))g'(x)$. For example if $F(x) = (5x + 1)^2$ then $g(x) = 5x + 1$ and $f(g(x)) = (g(x))^2$. By applying the chain rule, we find:

$$\begin{aligned}F'(x) &= 2(5x + 1)g'(x) \\ &= 2(5x + 1)5 \\ &= 50x + 10.\end{aligned}$$

$$\begin{aligned}F'(x) &= 2(5x + 1)g'(x) \\ &= 2(5x + 1)5 \\ &= 50x + 10.\end{aligned}$$

梯度是采用多个输入（或采用向量或某种其他复杂结构形式的一个输入）的函数导数的概括。函数的梯度是偏导数的向量。您可以将求函数的偏导数视为通过关注函数的输入之一并将所有其他输入视为常数值来求导数的过程。

Gradient is the generalization of derivative for functions that take several inputs (or one input in the form of a vector or some other complex structure). A gradient of a function is a vector of partial derivatives. You can look at finding a partial derivative of a function as the process of finding the derivative by focusing on one of the function’s inputs and by considering all other inputs as constant values.

例如，如果我们的函数定义为$f([x^{(1)},x^{(2)}]) = ax^{(1)} + bx^{(2)} + c$，然后函数的偏导数$f$ 关于 $x^{(1)}$，表示为$\frac{\partial f}{\partial x^{(1)}}$， 是（谁）给的，

For example, if our function is defined as $f([x^{(1)},x^{(2)}]) = ax^{(1)} + bx^{(2)} + c$, then the partial derivative of function $f$ with respect to $x^{(1)}$, denoted as $\frac{\partial f}{\partial x^{(1)}}$, is given by,

$$\frac{\partial f}{\partial x^{(1)}} = a + 0 + 0 = a,$$

$$\frac{\partial f}{\partial x^{(1)}} = a + 0 + 0 = a,$$

在哪里$a$是函数的导数$ax^{(1)}$;两个零点分别是$bx^{(2)}$和$c$， 因为$x^{(2)}$当我们计算关于的导数时被认为是常数$x^{(1)}$，并且任何常数的导数为零。

where $a$ is the derivative of the function $ax^{(1)}$; the two zeroes are respectively derivatives of $bx^{(2)}$ and $c$, because $x^{(2)}$ is considered constant when we compute the derivative with respect to $x^{(1)}$, and the derivative of any constant is zero.

类似地，函数的偏导数$f$关于$x^{(2)}$,$\frac{\partial f}{\partial x^{(2)}}$， 是（谁）给的，

Similarly, the partial derivative of function $f$ with respect to $x^{(2)}$, $\frac{\partial f}{\partial x^{(2)}}$, is given by,

$$\frac{\partial f}{\partial x^{(2)}} = 0 + b + 0 = b.$$

$$\frac{\partial f}{\partial x^{(2)}} = 0 + b + 0 = b.$$

函数的梯度$f$，表示为$\nabla f$由向量给出$\left[\frac{\partial f}{\partial x^{(1)}}, \frac{\partial f}{\partial x^{(2)}}\right]$。

The gradient of function $f$, denoted as $\nabla f$ is given by the vector $\left[\frac{\partial f}{\partial x^{(1)}}, \frac{\partial f}{\partial x^{(2)}}\right]$.

正如我在第 4 章中所说明的，链式法则也适用于偏导数。

The chain rule works with partial derivatives too, as I illustrate in Chapter 4.